{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3xW4O0zuKfWD"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "from openai import OpenAI\n",
        "import base64\n",
        "from google.colab import userdata\n",
        "from google.colab import drive"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "api_key = userdata.get('OPENAI_API_KEY')\n",
        "MODEL = \"gpt-4o-mini\"\n",
        "\n",
        "openai = OpenAI(api_key=api_key)"
      ],
      "metadata": {
        "id": "vUydnLeLKs03"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Sending images as input prompt and generating response based on the data inside thoese images**"
      ],
      "metadata": {
        "id": "blCx6gu1Vl46"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Chat Completion API\n",
        "\n",
        "https://platform.openai.com/docs/guides/images?api-mode=chat"
      ],
      "metadata": {
        "id": "5wlqKfVBKyjD"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "response = openai.chat.completions.create(\n",
        "    model=MODEL,\n",
        "    messages= [\n",
        "        { \"role\": \"user\", \"content\": [\n",
        "            {\"type\": \"text\", \"text\": \"What's in this image?\"},\n",
        "            {\n",
        "                \"type\": \"image_url\",\n",
        "                \"image_url\": {\n",
        "                    \"url\": \"https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg\",\n",
        "                },\n",
        "            },\n",
        "        ]}\n",
        "    ]\n",
        ")"
      ],
      "metadata": {
        "id": "dBrUkMCUKz0s"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(response.choices[0].message.content)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WItchxDLK2x6",
        "outputId": "4c02974d-8f87-4183-a09a-1a2635ec9d65"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "The image depicts a scenic landscape featuring a wooden pathway winding through a lush green field. The pathway extends into the distance, leading towards a horizon filled with greenery and trees under a blue sky adorned with some clouds. The scene conveys a peaceful and natural environment, ideal for walking or enjoying the outdoors.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Base64 encoded image"
      ],
      "metadata": {
        "id": "D8rAuv08V4K0"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Upload image in your googld drive\n",
        "# Connect this Colab to my Google Drive\n",
        "# It will ask permissions to connect your google drive with colab\n",
        "\n",
        "drive.mount(\"/content/drive\")\n",
        "image_path = \"/content/drive/MyDrive/Temp/car.jpg\""
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BY42-QT9W_2D",
        "outputId": "29a804b1-6d45-4fdc-f4e8-f50fe0df512d"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Mounted at /content/drive\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Function to encode the image\n",
        "def encode_image(image_path):\n",
        "    with open(image_path, \"rb\") as image_file:\n",
        "        return base64.b64encode(image_file.read()).decode(\"utf-8\")\n",
        "\n",
        "\n",
        "# If you run this example on your local machine then you just need provide image file path\n",
        "# Path to your image\n",
        "# image_path = \"./car.jpg\"\n",
        "\n",
        "# Getting the Base64 string\n",
        "base64_image = encode_image(image_path)\n",
        "\n",
        "completion = openai.chat.completions.create(\n",
        "    model=\"gpt-4o\",\n",
        "    messages=[\n",
        "        {\n",
        "            \"role\": \"user\",\n",
        "            \"content\": [\n",
        "                { \"type\": \"text\", \"text\": \"what's in this image?\" },\n",
        "                {\n",
        "                    \"type\": \"image_url\",\n",
        "                    \"image_url\": {\n",
        "                        \"url\": f\"data:image/jpeg;base64,{base64_image}\",\n",
        "                    },\n",
        "                },\n",
        "            ],\n",
        "        }\n",
        "    ],\n",
        ")\n",
        "\n",
        "print(completion.choices[0].message.content)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1fQ01A9TV5sW",
        "outputId": "90098080-744f-4210-8bf3-cdd99ea43b34"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "The image shows a sleek, high-performance sports car surrounded by dramatic lighting and colored smoke effects. The car has an aerodynamic design, featuring a prominent rear wing and stylish wheels. The lighting and smoke create a dynamic, almost cinematic atmosphere.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Responses API\n",
        "\n",
        "https://platform.openai.com/docs/guides/images?api-mode=responses"
      ],
      "metadata": {
        "id": "tCzSosP5K-JR"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Following values of property `type` are changed in Responses API\n",
        "\n",
        "text -> input_text\n",
        "\n",
        "image_url -> input_image\n",
        "\n",
        "```\n",
        "\"content\": [\n",
        "    { \"type\": \"input_text\", \"text\": \"what's in this image?\" },\n",
        "    {\n",
        "        \"type\": \"input_image\",\n",
        "        \"image_url\": f\"data:image/jpeg;base64,{base64_image}\",\n",
        "    },\n",
        "],\n",
        "```"
      ],
      "metadata": {
        "id": "gBzxAbx8XQKK"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "response = openai.responses.create(\n",
        "    model=MODEL,\n",
        "    input=[{\n",
        "        \"role\": \"user\",\n",
        "        \"content\": [\n",
        "            {\"type\": \"input_text\", \"text\": \"what's in this image?\"},\n",
        "            {\n",
        "                \"type\": \"input_image\",\n",
        "                \"image_url\": \"https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg\",\n",
        "            },\n",
        "        ],\n",
        "    }],\n",
        ")"
      ],
      "metadata": {
        "id": "2IPLSF30K--S"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(response.output_text)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "sejIL04-WVCe",
        "outputId": "0991813f-6ef5-4379-d5aa-2717a8e5f862"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "The image features a scenic natural landscape with a wooden path or boardwalk running through a lush green area. On either side of the path, there are tall grasses and some vegetation. The sky above is partly cloudy with a blue hue, indicating a bright day. This setting is likely near a wetland or grassy field, suggesting a tranquil outdoor environment.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Base64 encoded image"
      ],
      "metadata": {
        "id": "z1gX_OTVXTrB"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Google Drive is already mounted in above example"
      ],
      "metadata": {
        "id": "VdXysDIiYm2b"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Function to encode the image\n",
        "def encode_image(image_path):\n",
        "    with open(image_path, \"rb\") as image_file:\n",
        "        return base64.b64encode(image_file.read()).decode(\"utf-8\")\n",
        "\n",
        "\n",
        "# Path to your image\n",
        "# image_path = \"./car.jpg\"\n",
        "\n",
        "# Getting the Base64 string\n",
        "base64_image = encode_image(image_path)\n",
        "\n",
        "\n",
        "response = openai.responses.create(\n",
        "    model=\"gpt-4o\",\n",
        "    input=[\n",
        "        {\n",
        "            \"role\": \"user\",\n",
        "            \"content\": [\n",
        "                { \"type\": \"input_text\", \"text\": \"what's in this image?\" },\n",
        "                {\n",
        "                    \"type\": \"input_image\",\n",
        "                    \"image_url\": f\"data:image/jpeg;base64,{base64_image}\",\n",
        "                },\n",
        "            ],\n",
        "        }\n",
        "    ],\n",
        ")\n",
        "\n",
        "print(response.output_text)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "IZd8AWE0XX0q",
        "outputId": "6732bb46-79a1-4ac9-ced0-09b228529eee"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "The image shows a sleek, high-performance sports car enveloped in colorful smoke. The car is likely parked or moving slowly as the smoke swirls around it, creating a dramatic and dynamic effect. The lighting highlights the car's aerodynamic features and vibrant tail lights.\n"
          ]
        }
      ]
    }
  ]
}